The spectral transmittance spectrum is one of physical quantities representing a physical property specific to a subject of imaging. The spectral transmittance is a physical quantity representing a ratio of transmitted light to incident light at each wavelength and, unlike the color information such as an RGB value that depends on the change of illumination light, is the information specific to an object, whose values do not change depending on extrinsic influences. Thus the spectral transmittance is used in various fields as the information to reproduce the color of the subject itself. For example, in the field of pathology diagnosis that uses body tissue samples, particularly, pathological specimens, the spectral transmittance has been used, as an example of a spectral characteristic value, for analyzing images obtained by capturing samples. The examples of use of the spectral transmittance for pathology diagnosis will now be described below in detail.
As one of the pathological examinations for pathology diagnosis, the tissue diagnosis by which tissue is taken from a lesioned part and is observed with a microscope for diagnosing a disease or the degree of lesion expansion is known. This tissue diagnosis is also called “biopsy”, by which a block sample obtained by organ harvesting or a pathological specimen obtained by needle biopsy is sliced into several micrometers thick and is magnified with a microscope to obtain various findings, and this diagnosis has been widely used. Transmission observation using an optical microscope is one of the most common observation methods, because equipment and material are relatively inexpensive and easy to be handled, and the method has been used for many years. In the case of the transmission observation, the sliced samples absorb and scatter almost no light and are almost transparent and colorless. Thus, the samples are generally stained by a dye prior to observation.
Various staining methods have been proposed, and more than a hundred methods have been proposed in total. Particularly, for pathological specimens, hematoxylin-eosin staining (hereinafter referred to as “HE staining”) that uses bluish-purple hematoxylin and red eosin as pigment has been used as a standard staining method.
Hematoxylin is a natural substance extracted from plants, and has no stainability itself. However, hematin, an oxidative product of hematoxylin, is a basophilic dye and binds to a negatively charged substance. Deoxyribonucleic-acid (DNA) contained in cell nucleus is negatively charged by a phosphate group contained as a component and, thus is stained bluish-purple when it binds to hematin. As described above, a substance having stainability is its oxidative product, hematin, and is not hematoxylin. However, hematoxylin is commonly used as a name of dye, and this applies to the following explanations.
On the other hand, eosin is an acidophilic dye and binds to a positively charged substance. Amino acid and protein are charged negatively or positively depending on its pH environment, and have a strong tendency to be charged positively under acidity. Thus, there are some cases where acetic acid is added to eosin solution. The protein contained in cytoplasm is stained red or pale red when binding to eosin.
After being HE stained (a stained sample), cell nucleuses, bone tissues and the like are stained bluish-purple, and cytoplasm, connective tissue, blood cells and the like are stained red in the sample, which offers visibility. Thus an observer understands the size and positional relation and the like of elements constituting tissues such as cell nucleuses, thereby enabling the observer to determine a state of the sample morphologically.
In addition to a visual inspection by the observer, the stained sample can also be observed by multiband imaging displayed on a display screen of an external device. In the case where images are displayed on a screen, processing for estimating the spectral transmittance at each point of the sample from the obtained multiband image and processing for estimating dye amount of a dye that stains the sample based on the estimated spectral transmittance and the like are performed and, the image to be displayed, which is an RGB image of the sample for display, is composed.
Methods of estimating the spectral transmittance at each point of the sample from the multiband image of the sample include, for example, principal component analysis, Wiener estimation method and the like. The Wiener estimation is widely known as one of the linear filtering methods for estimating an original signal from an observed signal on which noise is superimposed and is a method for minimizing errors in view of the statistical properties of the observed object and the noise (observed noise). Signals from a camera contain some sort of noise. Thus the Wiener estimation is an extremely useful method for estimating an original signal.
A method of creating a virtual slide by composing a display image from a multiband image of a sample is described below. The virtual slide is an image created by patching one or more multiband images captured by a microscope device and, for example, an image created by patching a plurality of high-resolution images at each portion of a stained sample captured by a high-power microscope objective lens. The virtual slide means a wide-field and high-definition multiband image of the entire view of a stained sample.
First, a multiband image of a sample is captured, for example, based on a frame sequential method, while rotating a filter wheel to switch 16 pieces of bandpass filters. In this manner, multiband images having a pixel value of 16 bands can be obtained at each point of the sample. Although the dye is originally distributed three-dimensionally in a sample to be observed, the dye cannot be captured as a three-dimensional image as it is by using an ordinary transmission observation system, and the illumination light that has passed the sample is observed as a two-dimensional image projected on an imaging element of a camera. Therefore, each point described herein means a point at the sample corresponding to each pixel of the projected imaging element.
For an arbitrary point (pixel) x of a captured multiband image, a relation expressed by the following Equation (1) based on a response system of a camera is established between the pixel value g(x, b) in the band b and the spectral transmittance t(x, λ) at the corresponding point of the sample.g(x,b)=∫λƒ(b,λ)s(λ)e(λ)t(x,λ)dλ+n(b)  (1)
In the Equation (1), denotes wavelength, f(b,λ) denotes spectral transmittance of a “b”th filter, s(λ) denotes spectral sensitivity characteristic of a camera, e(λ) denotes spectral emission characteristic of illumination, and n(b) denotes observation noise in the band b. b denotes a serial number for identifying the band, and is an integer satisfying 1≦b≦16. In the practical calculation, the following Equation (2) obtained by discretizing the Equation (1) in a wavelength direction is used.G(x)=FSET(x)+N  (2)
In the Equation (2), when the number of sample points in a wavelength direction is designated as D and the number of bands is designated as B (in this case, B=16), G(x) denotes a matrix of B rows by one column corresponding to the pixel value g(x, b) at the point x. In the same manner, T(x) denotes a matrix of D rows by one column corresponding to t(x, λ), and F denotes a matrix of B rows by D columns corresponding to f(b, λ). On the other hand, S denotes a diagonal matrix of D rows by D columns, and a diagonal element corresponds to s(λ). In the same manner, E denotes a diagonal matrix of D rows by D columns, and a diagonal element corresponds to e(λ). N denotes a matrix of B rows by one column corresponding to n(b). In the Equation (2), because equations of a plurality of bands are put together using a matrix, a variable b representing a band is not described. Further, an integral of the wavelength λ is replaced by a product of matrices.
To simplify the description, the matrix H defined by the following Equation (3) is introduced. The matrix H is also called a system matrix.H=FSE  (3)
Thus, the Equation (2) is replaced by the following Equation (4).G(x)=HT(x)+N  (4)
Next, the spectral transmittance at each point of the sample is estimated from the captured multiband image by using the Wiener estimation. The estimation value of the spectral transmittance (spectral transmittance data), T^(x), can be calculated by the following Equation (5). T^ indicates that a symbol, “^(hat)”, representing an estimation value, is put over T.{circumflex over (T)}(x)=(x)  (5)
W is expressed by the following Equation (6), and is referred to as “Wiener estimation matrix” or “estimation operator used for the Wiener estimation”.W=RSSHt(RSSHt+RNN)−1  (6)Where, ( )t: transposed matrix, and ( )−1: inverse matrix.
In the Equation (6), RSS is a matrix of D rows by D columns and represents an autocorrelation matrix of spectral transmittance of the sample, and RNN is a matrix of B rows by B columns and represents an autocorrelation matrix of noise of the camera used for capturing.
In order to calculate an estimation operator W by which each main element such as cell nucleus, cytoplasm, blood cell, cavum and the like can appropriately be estimated, spectrum of each main element such as cell nucleus, cytoplasm, blood cell, cavum and the like are needed previously. Thus the user needs to measure the spectrum of each main element of the sample previously with a spectrometer while moving the measuring position, which may be troublesome.
In order to solve the above mentioned problem, for example, the Japanese Unexamined Patent Application Publication No. 2009-014354 discloses an image processing device by which an appropriate estimation operator, W, is calculated automatically. In this image processing device, a spectrum of each main element of a sample is measured with a spectrometer while moving the estimating position automatically, and an estimation operator W is calculated from the measured spectrum. Then, the estimation operator W is evaluated and if it is not appropriate, a spectrum of each main element of the sample is measured again and, thus an appropriate estimation operator W is calculated automatically.
According to the image processing device disclosed in the aforementioned document, an appropriate estimation operator W is calculated automatically, thereby reducing a burden on the user. However, the inventor of the present invention considers that, in the aforementioned image processing device, spectrum of a plurality of elements of a sample should be measured previously with a spectrometer while moving the measurement position. Thus the processing requires a lot of time and as a result, estimation of spectrum of the sample will be time-consuming.
In view of the aforementioned problem, the object of the present invention is to provide a virtual microscope system by which a stained sample image obtained by capturing the stained sample and statistical data of spectra can be obtained in a short period of time.